System and method of material handling using one or more imaging devices on the transferring vehicle to control the material distribution into the storage portion of the receiving vehicle

ABSTRACT

First imaging device collects first image data, whereas second imaging device collects second image data of a storage portion. A container identification module identifies a container perimeter of the storage portion in at least one of the collected first image data and the collected second image data. A spout identification module is adapted to identify a spout of the transferring vehicle in the collected image data. An image data evaluator determines whether to use the first image data, the second image data, or both based on an evaluation of the intensity of pixel data or ambient light conditions. An alignment module is adapted to determine the relative position of the spout and the container perimeter and to generate command data to the propelled portion to steer the storage portion in cooperative alignment such that the spout is aligned within a central zone or a target zone of the container perimeter.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is the US national phase application of PCTInternational Application PCT/US2013/025604, filed Feb. 11, 2013, titledMETHOD AND STEREO VISION SYSTEM FOR FACILITATING THE UNLOADING OFAGRICUTURUAL MATERIAL FROM A VEHICLE, which claims the priority of U.S.Provisional Application 61/597,346, filed Feb. 10, 2012, titled METHODAND STEREO VISION SYSTEM FOR FACILITATING THE UNLOADING OF AGRICUTURUALMATERIAL FROM A VEHICLE, which is incorporated by reference herein.

JOINT RESEARCH AGREEMENT

This application resulted from work performed under or related to ajoint research agreement between Carnegie Mellon University and Deere &Company, entitled “Development Agreement between Deere & Company andCarnegie Mellon University,” dated Jan. 1, 2008 and as such is entitledto the benefits available under 35 U.S.C. §103(c).

FIELD OF THE INVENTION

This invention relates to a method and stereo vision system forfacilitating the unloading of material from a vehicle.

BACKGROUND

Certain prior art systems may attempt to use global positioning system(GPS) receivers to maintain proper spacing between two vehicles duringthe unloading or transferring of agricultural material or othermaterial, such as coal and other minerals, between the vehicles.However, such prior art systems are susceptible to misalignment of theproper spacing because of errors or discontinuities in the estimatedposition of the GPS receivers. For example, one or more of the GPSreceivers may misestimate its position because of electromagneticinterference, multipath propagation of the received satellite signals,intermittent reception of the satellite signals or low received signalstrength of the satellite signals, among other things. If the vehiclesuse cameras or other imaging devices in an outdoor work area, such as anagricultural field, the imaging devices may be subject to transitorysunlight, shading, dust, reflections or other lighting conditions thatcan temporarily disrupt proper operation of the imaging devices; hence,potentially produce errors in estimated ranges to objects observed bythe imaging devices. Thus, there is a need for an improved system formanaging the unloading of agricultural material from a vehicle tocompensate for or address error in the estimated positions or alignmentof the vehicles.

SUMMARY OF THE INVENTION

The system and method facilitates the transfer of agricultural materialfrom a transferring vehicle (e.g., harvesting vehicle) to a receivingvehicle (e.g., grain cart). The system and method comprises a receivingvehicle, which has a propelled portion for propelling the receivingvehicle and a storage portion for storing agricultural material and atransferring vehicle for transferring harvested agricultural materialinto the storage portion of the receiving vehicle.

Two embodiments of the present invention include one or two imagingdevices on only the transferring vehicle, either a combine or aself-propelled forge harvester. A first embodiment mounts one or twoprimary imaging device on the combine (a transferring vehicle) and noimaging devices mounted on the receiving vehicle. A second embodimentmounts one or two imaging devices on the self-propelled forge harvester,also a transferring vehicle, and no imaging devices on the receivingvehicle.

Embodiments of the present invention include a first imaging device thatis mounted at a first location on the receiving vehicle and facestowards the storage portion of the receiving vehicle. The first imagingdevice collects first image data. A second imaging device is associatedwith a second location on (e.g., mounted on or movably attached to) thetransferring vehicle and faces towards the storage portion of thereceiving vehicle. The second imaging device collects second image data.

The transferring vehicle of any of the above mentioned embodiments caninclude as an image processing module having a container or binidentification module that can identify a container or bin perimeter ofthe storage portion in at least one of the collected first image dataand the collected second image data (where a second imaging device isincorporated into the system configuration). The image processing canalso include a spout localizer that is adapted to identify a spout ofthe transferring vehicle in the collected image data (collected firstimage data, collected second image data, or both). The image processingmodule can include an image data evaluator that determines whether touse the first image data, the second image data or both (where a secondimaging device is incorporated into the system configuration), based onan evaluation of material variation of intensity of pixel data ormaterial variation in ambient light conditions during a sampling timeinterval. In a system with only one imaging device, the image dataevaluator is either not activated, is not incorporated into the system,or includes logic that passes the only collected image to the nextfunction. The image processing module can also include an alignmentmodule that is adapted to determine the relative position of the spoutand the container perimeter, and to generate command data to thesteering controller of the transferring vehicle to steer thetransferring vehicle in cooperative alignment with the receiving vehiclesuch that the spout is aligned within a central zone (or other targetzone) of the container perimeter.

In operation, a method for facilitating the transfer of material from atransferring vehicle having a material distribution end to a receivingvehicle having a bin to the store transferred material, the methodcomprising the steps of:

a. identifying and locating the bin;

b. detecting a representation of the fill level or volumetricdistribution of the material in the bin;

c. aligning the material distribution end over a current target area ofthe bin requiring the material (wherein a current target area can be aninitial target area the material distribution end is positioned when thefilling of material begins);

d. determining subsequent target areas of the bin that require materialbased on the representation of the fill level or volumetric distributionof the material in the bin and a desired fill pattern (such asfront-to-back, back-to-front, center-to-front-to-back,center-to-back-to-front) to fill the bin;

e. transferring the material from the transferring vehicle to thecurrent target area of the bin of the receiving vehicle;

f. detecting when the current target area of the bin is filled with thematerial;

g. repeating steps c-f until the subsequent target areas of the bin arefilled; and

h. terminating the transfer of the material from the transferringvehicle to the receiving vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of one embodiment of a machinevision-augmented guidance system for a transferring vehicle being acombine for facilitating the unloading of agricultural material from thetransferring vehicle (e.g., combine);

FIG. 2 is a block diagram of another embodiment of a machinevision-augmented guidance for a transferring vehicle being aself-propelled forge harvester for facilitating the unloading ofagricultural material from the transferring vehicle;

FIG. 3 illustrates a top view of an imaging devices mounted on atransferring vehicle and facing toward a receiving vehicle;

FIG. 4A illustrates a top view of imaging devices (e.g., a monocular orstereo vision system) mounted on a transferring vehicle and facing astorage portion of the receiving vehicle;

FIG. 4B illustrates a top view of imaging devices (e.g., a monocular orstereo vision system) mounted on a receiving vehicle and a transferringvehicle that face a storage portion of a receiving vehicle;

FIG. 4C illustrates a view in a horizontal plane as viewed alongreference line 4C-4C in FIG. 4B;

FIG. 4D illustrates a two-dimensional representation of various possibleillustrative distributions of material in the interior of a container(or bin) or storage portion, consistent with a cross-sectional viewalong reference line 4D-4D in FIG. 4B;

FIG. 4E is a top view of a transferring vehicle and a receiving vehicle,where the transferring vehicle is aligned within a matrix of possibleoffset positions;

FIG. 5A illustrates a block diagram of a container identificationprocess using rectified images;

FIG. 5B illustrates a block diagram of a container identificationprocess capable of using rectified images and disparity images;

FIG. 6A is a block diagram of a spout localizing process using rectifiedimages and spout position data;

FIG. 6B is a block diagram of a spout localizing process using rectifiedimages, disparity images, and spout position data;

FIG. 7 is a flow chart of a method for operating a machinevision-augmented guidance system for facilitating the unloading ofagricultural material from a transferring vehicle; and

FIG. 8 is a schematic illustrating the data flow and processing by theimage processing module from raw images to vehicle commands.

DETAILED DESCRIPTION OF THE INVENTION

In accordance with one embodiment of the present invention that requiresimaging devices on the transferring vehicle, FIGS. 1 and 2 show machinevision augmented guidance systems 11, 111 for a transferring vehicle 91(FIG. 3) for managing the unloading of agricultural material (e.g.,grain) from the transferring vehicle 91 (FIG. 1—combine; FIG.2—self-propelled forge harvester) to a receiving vehicle 79 (e.g., graincart or wagon). The transferring vehicle 91 may comprise other vehiclessuch as a harvester or other heavy equipment that collects or harvestsmaterial for transfer to the receiving vehicle. For example, a stereoimaging system augments satellite navigation receivers orlocation-determining receivers 42 for guidance of the transferringvehicle 91. Now turning to FIG. 3, the first imaging device 10 has afirst field of view 77, indicated by the dashed lines. The secondimaging device 12 has a second field of view 177, indicated by thedashed lines. The boundaries of the fields of view 77, 177 are merelyshown for illustrative purposes and will vary in actual practice. Thesystems 11, 111 can comprises a first imaging device 10 and a secondimaging device 12 coupled to an image processing module 18 (FIGS. 1 and2). Embodiments of first imaging device 10 may comprise a primary stereocamera or a monocular camera, while the second imaging device 12 maycomprise a secondary stereo camera or a monocular camera. In oneconfiguration, the second imaging device 12 is a stereo camera and canbe optional and provides redundancy to the first imaging device 10 incase of failure, malfunction or unavailability of image data from thefirst imaging device 10 when the first field of view 77 of the firstimaging device 10 is sufficient to view within container 85. In oneconfiguration, the second imaging device is monocular and is requiredfor a stereo image of the container or bin 85 when used in conjunctionwith an image from a monocular first imaging device 10 with the firstfield of view 77 sufficient to view within container 85. FIG. 3illustrates a plan view of a transferring vehicle 91 and a receivingvehicle 79. As illustrated in FIG. 3 for explanatory purposes, thetransferring vehicle 91 is shown as a combine with a harvesting head185, whereas the receiving vehicle 79 is shown as a tractor and a graincart. More generally, the receiving vehicle 79 comprises the combinationof a propulsion unit 75 and a storage unit 93 (e.g., a towed storageunit). Each imaging device 10, 12 includes an image rectifier 101 totransform the raw image into a rectified image. Though the example oftransferred material disclosed herein is agricultural material, theinvention is not to be limited to agricultural material and isapplicable to other materials such as coal and other minerals.

FIG. 3 shows a first imaging device 10 on the transferring vehicle 91(e.g., combine) and a second imaging device 12 on a spout 89 of thetransferring vehicle 91. The second imaging device 12 can be optional ifthe first imaging device 10 is a stereo camera and the first field ofview 77 of the first imaging device 10 is sufficient to view withincontainer 85. The spout 89 may also be referred to as an unloadingauger. The spout end 87 may be referred to as a boot. In FIG. 3, thespout 89, or the spout end 87, is generally aligned over a central zone83, central region or target area with the grid pattern 82 (FIGS. 4A and4B) of the storage container 85 of the receiving vehicle 79 forunloading material from the transferring vehicle 91 to the receivingvehicle 79. Similarly, the transferring vehicle 91 and the receivingvehicle 79 are aligned in position as shown, regardless of whether thevehicles move together in a forward motion (e.g., with coordinated ortracked vehicle headings) during harvesting, as is typical, or arestationary. During unloading, the master controller 59 (FIGS. 1 and 2)facilitate maintenance of a generally uniform spatial offset (e.g., agenerally static offset that varies only within a predetermined targettolerance) between the vehicles 91, 79, subject to any incrementaladjustment of the offset for uniform filling of the container 85. Themaster controller 59 supports maintenance of a uniform fore/aft offset(Φ or φ) and a lateral offset (Δ).

Now returning to FIGS. 1, 2, and 3, the transferring vehicle 91 may beequipped with a spout rotation sensor 116 to measure the rotation angleof the spout 89. For a spout-mounted imaging device 12, the rotationangle of the spout 89 may be used to facilitate fusion of image datafrom the first imaging device 10 and the second imaging device 12, or toconstruct stereo image data where the first imaging device 10 and thesecond imaging device 12 individually provide monocular image data forthe same scene or object.

In any arrangement of imaging devices 10, 12 disclosed herein where thefields of view 77, 177 overlap, data fusion of image data from a firstimaging device 10 and a second imaging device 12 enables the imageprocessing module 18 to create a virtual profile of the materialdistribution level (FIG. 4D) inside the storage portion 85, even whenthe entire surface of the agricultural material is not visible to one ofthe two imaging devices 10, 12. Even if the second imaging device 12 isnot mounted on the spout 89 in certain configurations, the spoutrotation sensor 116 may facilitate using the spout end 87 as a referencepoint in any collected image data (e.g., for fusion, virtual stitchingor alignment of image data from different imaging devices.) The virtualprofile of the entire surface of the agricultural material in thestorage portion 93 enables the systems 11, 111 or imaging module 18 tointelligently execute a fill strategy for the storage portion 93 of thereceiving vehicle 79.

The first imaging device 10 and the second imaging device 12 may providedigital data format output as stereo video image data or a series ofstereo still frame images at regular or periodic intervals, or at othersampling intervals. Each stereo image (e.g., the first image data or thesecond image data) has two component images of the same scene or aportion of the same scene. For example, the first imaging device 10 hasa first field of view 77 of the storage portion 93 of the receivingvehicle 79, where the first field of view 77 overlaps at least partiallywith a second field of view 177 of the second imaging device 12 (ifpresent). In one embodiment, the first imaging device 10, the secondimaging device 12, or both may comprise a charge-coupled device (CCD), acomplementary metal-oxide semiconductor (CMOS) array, or anothersuitable device for detection or collection of image data.

In one configuration, an optical sensor 110, 112 comprises a lightmeter, a photo-sensor, photo-resistor, photo-sensitive device, or acadmium-sulfide cell. A first optical sensor 110 may be associated withthe first imaging device 10; a second optical sensor 112 may beassociated with the second imaging device 12. The first optical sensor110 and the second optical sensor 112 each may be coupled to the imageprocessing module 18. The optical sensor 110, 112 provides a reading orlevel indicative of the ambient light in the field of view of itsrespective imaging device 10, 12.

The image processing module 18 may be coupled, directly or indirectly,to lights 14 (FIG. 1) on a transferring vehicle 91 for illumination of astorage container 85 and/or spout 89. For example, the image processingmodule 18 may include a light controller 50 (FIG. 2) that comprisescontrol drivers, relays or switches, which in turn control theactivation or deactivation of lights 14 on the transferring vehicle 91.The image processing module 18 may activate the lights 14, 52 on thetransferring vehicle for illumination of the storage container 85 (FIG.3A), spout 89 or both if an optical sensor 110, 112 or light meterindicates that an ambient light level is below a certain minimumthreshold. In one configuration the optical sensor 110, 112 face towardthe same direction as the lens or aperture of the imaging devices 10,12.

In the combine embodiment (FIG. 1), vehicle controller 46 controls spout89 that includes a rotation sensor 116 for sensing a spout rotationangle (α) in FIG. 5A and (β) in FIG. 5C of the spout 89 with respect toone or more axes of rotation, and a rotation actuator 216 for moving thespout 89 to change the spout rotation angle; hence, the spout 89position with respect to the receiving vehicle 79 or its storagecontainer 85. The rotation actuator 216 may comprise a motor, a linearmotor, an electro-hydraulic device, a ratcheting or cable-actuatedmechanical device, or another device for moving the spout 89, or thespout end 87. The spout rotation angle may comprise a simple angle, acompound angle or multi-dimensional angles that is measured withreference to a reference axis parallel to the direction of travel of thetransferring vehicle.

If the rotation actuator 216 comprises an electro-hydraulic device, theuse of proportional control valves in the hydraulic cylinder of theelectro-hydraulic device that rotates the spout (or changes the spoutrotation angle) facilitates finer adjustments to the spout angle (e.g.,a) than otherwise possible. Accordingly, proportional control valves ofthe electro-hydraulic device support rotation actuator 216 for an evenprofile or distribution of unloaded agricultural material within thestorage portion 93 or container or bin 85. Many commercially availablecombines are typically equipped with non-proportional control valves forcontrolling spout angle or movement of the spout 89; electro-hydraulicdevices with non-proportional control valves can fill the storagecontainer with an inefficient multi-modal or humped distribution (e.g.,508) of agricultural material with local high areas and local low areas,as depicted in FIG. 4D, for example.

A vehicle controller 46 may be coupled to the vehicle data bus 60 toprovide a data message that indicates when the auger drive 47 forunloading agricultural material from the transferring vehicle isactivated and inactive. The auger drive 47 may comprise an auger, anelectric motor for driving the auger, and a rotation sensor for sensingrotation or rotation rate of the auger or its associated shaft. In oneembodiment, the auger (not shown) is associated with a container forstoring agricultural material (e.g., a grain tank) of a transferringvehicle 91. If the vehicle controller 46 (e.g., auger controller)indicates that the auger of the transferring vehicle 91 is rotating oractive, the imaging processing module 18 activates the spout localizer22 and container or bin identification module 20. Thus, vehiclecontroller 46 may conserve data processing resources or energyconsumption by placing the container identification module 20 and thespout identification module 22 in an inactive state (or standby mode)while the transferring vehicle 91 is harvesting, but not unloading, theagricultural material to the receiving vehicle 79.

In FIG. 1, the imaging processing module 18 or any other controller maycomprise a controller, a microcomputer, a microprocessor, amicrocontroller, an application specific integrated circuit, aprogrammable logic array, a logic device, an arithmetic logic unit, adigital signal processor, or another data processor and supportingelectronic hardware and software. In one embodiment, the imageprocessing module 18 comprises a disparity generator 103, a containeridentification module 20, a spout localizer 22, an alignment module 24,a material profile module 27, and a vehicle model 1000.

The image processing module 18 may be associated with a data storagedevice may comprise electronic memory, non-volatile random accessmemory, a magnetic disc drive, an optical disc drive, a magnetic storagedevice or an optical storage device, for example. If the containeridentification module 20, the spout localizer 22, the alignment module24, material profile module 27, and vehicle model 1000, are softwaremodules they are stored within the data storage device.

The container identification module 20 identifies a set oftwo-dimensional or three dimensional points (e.g., in Cartesiancoordinates or Polar coordinates) in the collected image data or in thereal world that define at least a portion of the container perimeter 81of the storage portion 85 (FIG. 3). The set of two-dimensional or threedimensional points correspond to pixel positions in images collected bythe first imaging device 10, the second imaging device 12, or both. Thecontainer identification module 20 may use or retrieve containerreference data.

The container reference data comprises one or more of the following:reference dimensions (e.g., length, width, height), volume, referenceshape, drawings, models, layout, and configuration of the container 85,the container perimeter 81, the container edges 181; referencedimensions, reference shape, drawings, models, layout, and configurationof the entire storage portion 93 of receiving vehicle; storage portionwheelbase, storage portion turning radius, storage portion hitchconfiguration of the storage portion 93 of the receiving vehicle; anddistance between hitch pivot point and storage portion wheelbase. Thecontainer reference data may be stored and retrieved from the datastorage device (e.g., non-volatile electronic memory). For example, thecontainer reference data may be stored by, retrievable by, or indexed bya corresponding receiving vehicle identifier in the data storage deviceof the transferring vehicle systems 11, 111. For each receiving vehicleidentifier, there can be a corresponding unique container reference datastored therewith in the data storage device.

In one configuration, the container identification module 18 identifiesthe position of the container or bin 85 as follows. If the linearorientation of a set of pixels in the collected image data conforms toone or more edges 181 of the perimeter 81 of the container 85 asprescribed by the container reference data, the position of thecontainer 85 has been identified. A target zone, central region orcentral zone of the container opening 83 of the container 85 can beidentified by dividing (by two) the distance (e.g., shortest distance orsurface normal distance) between opposite sides of the container, or byidentifying corners of the container and where diagonal lines thatintercept the corners intersect, among other possibilities. In oneconfiguration, the central zone may be defined as an opening (e.g.,circular, elliptical or rectangular) in the container with an openingsurface area that is greater than or equal to the cross-sectionalsurface area of the spout end by a factor of at least two, althoughother surface areas fall within the scope of the claims.

The spout localizer 22 identifies one or more of the following: (1) thespout pixels on at least a portion of the spout 89, or (2) spout endpixels that are associated with the spout end 87 of the spout 89. Thespout identification module 22 may use color discrimination, intensitydiscrimination, or texture discrimination to identify background pixelsfrom one or more selected spout pixels with associated spout pixelpatterns or attributes (e.g., color or color patterns (e.g., Red GreenBlue (RGB) pixel values), pixel intensity patterns, texture patterns,luminosity, brightness, hue, or reflectivity) used on the spout 89 or onthe spout end 87 of the spout 89 for identification purposes.

The alignment module 24, the master controller 59, or both estimate ordetermine motion commands at regular intervals to maintain alignment ofthe spout 56 over the central zone, central region or target of thecontainer 85 for unloading agricultural material. The alignment module24, the master controller 59, or both, may send commands or requests tothe transferring vehicle 91 with respect to its speed, velocity orheading to maintain alignment of the position of the transferringvehicle 91 with respect to the receiving vehicle 79. For example, thealignment module 24 may transmit a request for a change in a spatialoffset between the vehicles 79, 91 to the master controller 59. Inresponse, the master controller 59 or the coordination module 57transmits a steering command or heading command to the steeringcontroller 32, a braking or deceleration command to a braking system 34,and a propulsion, acceleration or torque command to a propulsioncontroller 40 to achieve the target spatial offset or change in spatialoffset.

In another configuration, the alignment module 24 may regularly orperiodically move, adjust or rotate the target zone or central zoneduring loading of the container 85 of the receiving vehicle to promoteeven filling, a uniform height, or uniform distribution of theagricultural material in the entire container 85, where the imageprocessing module 18 identifies the fill state of the agriculturalmaterial in the image data from the material profile module 27.

The imaging module 18 may comprise material profile module 27 or a filllevel sensor for detecting a one-dimensional, two-dimensional orthree-dimensional representation of the fill level or volumetricdistribution of the agricultural material in the container 85 or storageportion 93. For example, FIG. 4D shows various illustrativetwo-dimensional representations of the fill state of the container 85,or the distribution of agricultural material in the container 85, whereFIG. 4D will be described later in detail.

In one configuration, the coordination module 57 or the steeringcontroller 32 adjusts the relative position (of offset) of thetransferring vehicle 91 to the receiving vehicle 79. The alignmentmodule 24, the coordination module 57 and the auger rotation system 116may control the relative position of the spout 89 or the spout end 87 tothe container perimeter 81 to achieve an even fill to the desired filllevel. For example, rotator actuator 216 of the combine may adjust thespout angle (e.g., a first spout angle (α), a second spout angle (β) ora compound angle (α and β)) that the spout 89 makes with respect to areference axis or reference coordinate system associated with thetransferring vehicle 91 or a generally vertical plane associated withthe direction of travel of the transferring vehicle 91, where the spout89 meets and rotates with respect to the vehicle. With regards to theself-propelled forge harvester, the spout angle is controlled by spoutcontroller 54 in communication with rotation sensor 116, tilt sensor118, deflector sensor 120, rotation actuator 122, tilt actuator 124, anddeflector actuator 126.

The spout end 87 may be adjusted for unloading agricultural material byshifting its spout angle or spout position, within the containerperimeter 81 and a tolerance clearance from the container perimeter 81within the container 85. The spout end 87 may be adjusted by varioustechniques that may be applied alternately, or cumulatively. Under afirst technique, the alignment module 24 adjusts the spout end 87 forunloading agricultural material by shifting its spout angle (e.g., afirst spout angle (α), a second spout angle (β), or both (α and β).Under a second technique, the alignment module 24 requests (or commands)the coordination module 57 to adjust the fore/aft offset adjustment (Φor φ), the lateral adjustment (Δ), or both, where the coordinationmodule 57 manages or choreographs the relative fore/aft offset andlateral offset between the transferring vehicle 91 and receiving vehicle79. Under a third technique, the alignment module 24 primarily adjuststhe spout end 87 for unloading agricultural material by shifting itsspout angle and the coordination module 57 secondarily and regularly(e.g., periodically) moves the fore/aft offset and the lateral offset byfore/aft offset adjustment (Φ or φ), the lateral adjustment (Δ),respectively, to achieve a uniform fill state or level loading of thecontainer with the agricultural material. Accordingly, the spout end 87may be adjusted regularly (e.g., in a matrix of one or more rows orcolumns of preset offset positions) for unloading agricultural materialby shifting the spatial relationship between the transferring vehicleand the receiving vehicle by a fore and aft offset or a lateral offsetto achieve a target alignment or desired even distribution of fillingthe container 85 or storage portion 93 with agricultural material (FIG.4E), while using the spout angle adjustment for fine tuning of thedistribution of the agricultural material within the container (e.g.,from each position within the matrix).

In the image processing module 18, the image data evaluator 25 comprisean evaluator, a judging module, Boolean logic circuitry, an electronicmodule, a software module, or software instructions for determiningwhether to use the first image data, the second image data, or both foralignment of a relative position of the spout and the containerperimeter (or alignment of the spatial offset between the vehicles)based on evaluation of material variation of intensity of pixel data ormaterial variation in ambient light conditions during a sampling timeinterval.

In the combine, master controller 59 is coupled to the vehicle data bus(e.g., 60). Whereas in the self-propelled forge harvester, mastercontroller 59 is coupled to the implement data base 58 that is connectedto vehicle data bus 60 via gateway 29. In one embodiment, the mastercontroller 59 comprises an auto-guidance module 55 and coordinationmodule 57. The auto-guidance module 55 or master controller 59 cancontrol the transferring vehicle 91 in accordance with location datafrom the first location determining receiver 42 and a path plan ordesired vehicle path (e.g., stored in data storage). The auto-guidancemodule 55 or master controller 59 sends command data to the steeringcontroller 32, the braking controller 36 and the propulsion controller40 to control the path of the transferring vehicle 91 to trackautomatically a path plan or to track manually steered course of anoperator via the user interface 44 or steering system 30.

The coordination module 57 may facilitate alignment of movement (e.g.,choreography) between the transferring vehicle 91 and the receivingvehicle 79 during unloading or transferring of agricultural materialbetween the vehicles. For example, the coordination module 57 mayfacilitate maintenance of a uniform lateral offset (Δ) and a uniformfore/aft offset (Φ or φ) between the vehicles 91, 79 during unloading ofthe agricultural material, subject to any adjustments for attainment ofa uniform distribution of material in the container 85. Collectively,the uniform lateral offset and uniform for/aft offset may be referred toas a uniform spatial offset. In certain embodiments, maintenance of thelateral offset and fore/aft offset, or coordination of any shift in thelateral offset and fore/aft offset (e.g., pursuant to a two-dimensionalmatrix of pre-established positions (x, y points) for uniform loading ofa respective particular container or storage portion), is a necessary ordesired precondition to implementing spout angle adjustment of the spout89 or spout end 87 by the alignment module 24.

In one embodiment in a leader mode, the transferring vehicle 91 issteered by the auto-guidance module 55 or the steering controller 32 inaccordance with path plan, or by a human operator. If the transferringvehicle 91 operates in an automated mode or auto-steering mode, themaster controller 59 provides command data locally to the steeringcontroller 32, braking controller 36, and propulsion engine controller40 of the transferring vehicle 91. In an automated mode and in aleader-follower mode, the transferring vehicle 91 is steered and alignedautomatically during transfer of agricultural material from thetransferring vehicle 91 to the receiving vehicle 79.

The image processing module 18 provides image data (rectified,disparity, or both) to a user interface processing module 26 thatprovides, directly or indirectly, status message data and performancemessage data to a user interface 44.

In one embodiment, a location determining receiver 42, a first wirelesscommunications device 48, a vehicle controller 46, a steering controller32, a braking controller 36, and a propulsion controller 40 are capableof communicating over the vehicle data bus 60. In turn, the steeringcontroller 32 is coupled to a steering system 30 of the transferringvehicle 91; the braking controller 36 is coupled to the braking system34 of the transferring vehicle 91; and the propulsion controller 40 iscoupled to the propulsion system 38 of the transferring vehicle 91.

The steering system 30 may comprise an electrically-driven steeringsystem, an electro-hydraulic steering system, a gear driven steeringsystem, a rack and pinion gear steering system, or another steeringsystem that changes the heading of the transferring vehicle 91 or one ormore wheels of the transferring vehicle 91. The braking system 34 maycomprise a regenerative braking system, an electro-hydraulic brakingsystem, a mechanical breaking system, or another braking system capableof stopping the vehicle by hydraulic, mechanical, friction or electricalforces. The propulsion system 38 may comprise one or more of thefollowing: (1) the combination of an electric motor and an electriccontroller, (2) internal combustion engine that is controlled by anelectronic fuel injection system or another fuel metering device thatcan be controlled by electrical signals, or (3) a hybrid vehicle inwhich an internal combustion engine drives a electrical generator, whichis coupled to one or more electric drive motors.

The systems 11, 111 facilitate the transfer of agricultural materialfrom the transferring vehicle 91 to a receiving vehicle 79 with apropelled portion 75 for propelling the receiving vehicle and a storageportion 93 for storing agricultural material. A stereo imaging device,such as the first imaging device 10, faces towards the storage portion93 of the receiving vehicle 79. As shown in FIGS. 1 and 2, the firstimaging device 10 and the optional second imaging device 12 are mountedon the transferring vehicle 91, consistent with FIG. 3.

In summary, one or more imaging devices 10, 12 are arranged to collectimage data. A container identification module 20 identifies a containerperimeter 81 of the storage portion 93 in the collected image data. Thestorage portion 93 has an opening inward from the container perimeterfor receipt of the agricultural material. A spout localizer 22 isconfigured to identify a spout 89 of the transferring vehicle 91 in thecollected image data. An alignment module 24 is adapted for determiningthe relative position of the spout 89 and the container perimeter 81 andfor generating command data to the transferring vehicle 91 to steer thetransferring vehicle 91 in cooperative alignment with receiving vehicle79 such that the spout 89 is aligned within a central zone 83 or openingof grid pattern 82 of the container perimeter 81. A steering controller32 is associated with a steering system 30 of the transferring vehicle91 for steering the transferring vehicle 91 in accordance with thecooperative alignment.

In one embodiment, an optional mast controller 674, indicated by dashedlines, is coupled to the vehicle data bus 60 (FIG. 1) or the implementdata bus 58 (FIG. 2) to control an optional adjustable mast 573 formounting and adjustably positioning the first imaging device 10, thesecond imaging device 12, or both. The mast controller 674 is adapted tochange the orientation or height above ground of the first imagingdevice 10, the second imaging device 12 or both, where the orientationmay be expressed as any of the following: a tilt angle, a pan angle, adown-tilt angle, a depression angle, or a rotation angle.

In one illustrative embodiment of a machine-vision guidance system 11,111 that has an adjustable mast 573, at least one imaging device 10, 12faces towards the storage portion 93 of the receiving vehicle 79 andcollects image data. For example, via data from the mast controller 674the adjustable mast 573 is capable of adjusting a height of the imagingdevice 10, 12 within a height range, adjusting a down-tilt angle of theimaging device 10, 12 within a down-tilt angular range, and a rotationalangle or pan angle within a pan angular range. The image processingmodule 18 is adapted or programmed (e.g., with software instructions orcode) to determine whether to adjust the height of the imaging device10, 12 or whether to decrement or increment the down-tilt angle of theimaging device 10, 12 based on evaluation of material variation ofintensity of pixel data or material variation in ambient lightconditions (e.g., from the optical sensor 110, 112) during a samplingtime interval. Under certain operating conditions, such as outdoorambient light conditions, increasing or incrementing the down-tilt anglemay increase the quality level of the collected image data or reducevariation in the intensity of the image data to below a thresholdvariation level. Reduced variation in intensity of the image data orreduced collection of dust or debris on a lens of the imaging device aresome advantages that can be realized by increasing or adjustingdown-tilt angle of the imaging device 10, 12, for example. As previouslynoted, a container identification module 20 can identify a containerperimeter 81 of the storage portion 93 in the collected image data.Similarly, a spout localizer 22 can identify a spout of the transferringvehicle 91 in the collected image data. An alignment module 24determines the relative position of the spout 89 and the containerperimeter 81 and generates command data to the steering controller 32 tosteer the transferring vehicle 91 in cooperative alignment with thereceiving vehicle 79 such that the spout 89, or spout end 87, is alignedwithin a target zone of the grid pattern 82 or central zone 83 of thecontainer perimeter 81.

In one illustrative embodiment of a machine-vision guidance system withthe adjustable mast 573, the image processing module 18 sends a datamessage to a mast controller 674 (or the adjustable mast 573) toincrement or increase the down-tilt angle if the material variation ofintensity of pixel data or if the material variation in ambient lightconditions exceeds a threshold variation level during a sampling timeinterval. For example, the image processing module 18 sends a datamessage to a mast controller 674 to increment or increase the down-tiltangle at discrete levels (e.g., one degree increments or decrements)within an angular range of approximately negative ten degrees toapproximately negative twenty-five degrees from a generally horizontalplane.

In one configuration, a user interface 44 is arranged for enteringcontainer reference data or dimensional parameters related to thereceiving vehicle. For example, the container reference data ordimensional parameters comprise a distance between a trailer hitch orpivot point (which interconnects the propulsion unit 75 and the storageportion 93) and front wheel rotational axis of the storage portion 93 ofthe receiving vehicle 79.

In an alternate embodiment, the first imaging device 10 comprises amonocular imaging device and the second imaging device 12 comprises amonocular imaging device that provides first monocular image data andsecond monocular image data, respectively. The image processing module18 can create a stereo image from the first monocular image data (e.g.,right image data) and the second monocular image data (e.g., left imagedata) with reference to the relative position and orientation of thefirst imaging device 10 and the second imaging device 12. The imageprocessing module 18 determines: (1) at least two points on a commonvisual axis that bisects the lenses of both the first imaging device 10and the second imaging device 12, and (2) a linear spatial separationbetween the first imaging device 10 and the second imaging device 12,where the first field of view 477 (FIG. 4A) of the first imaging device10 and the second field of view 277 (FIG. 4A) of the second imagingdevice 12 overlap, at least partially, to capture the spout 89, thespout end 87 and the container perimeter 81 in the collected image data.

In an alternate embodiment, FIGS. 1 and 2 further comprises an optionalodometer sensor 440, and an optional inertial sensor 442, as illustratedby the dashed lines. The odometer sensor 440 may comprise a magneticrotation sensor, a gear driven sensor, or a contactless sensor formeasuring the rotation of one or more wheels of the transferring vehicleto estimate a distance traveled by the transferring vehicle during ameasurement time period, or a ground speed of the transferring vehicle.The odometry sensor 440 may be coupled to the vehicle data bus 60 or animplement data bus 58. The inertial sensor 442 may comprise one or moreaccelerometers, gyroscopes or other inertial devices coupled to thevehicle data bus 60 or an implement data bus 58. The optional odometrysensor 440 and the optional inertial sensor 442 may augment orsupplement position data or motion data provided by the first locationdetermining receiver 42.

The vision-augmented guidance system 111 of FIG. 2 is similar to thesystem 11 of FIG. 1; except that the system 111 of FIG. 2 furthercomprises an implement data bus 58, a gateway 29, and light controller50 and spout controller 54 coupled to the vehicle data bus 60 for thelights 14 and spout 89, respectively. The light controller 50 controlsthe lights 14; the spout controller 54 controls the spout 89 via aservo-motor, electric motor, or an electro-hydraulic mechanism formoving or adjusting the orientation or spout angle of the spout 89, orits spout end 87. In one configuration, the implement data bus 58 maycomprise a Controller Area Network (CAN) implement data bus. Similarly,the vehicle data bus 60 may comprise a controller area network (CAN)data bus. In an alternate embodiment, the implement data bus 58, thevehicle data bus 60, or both may comprise an ISO (InternationalOrganization for Standardization) data bus or ISOBUS, Ethernet oranother data protocol or communications standard.

The self-propelled forge harvester includes gateway 29 to support secureor controlled communications between the implement data bus 58 and thevehicle data bus 60. The gateway 29 comprises a firewall (e.g., hardwareor software), a communications router, or another security device thatmay restrict or prevent a network element or device on the implementdata bus 58 from communicating (e.g., unauthorized communication) withthe vehicle data bus 60 or a network element or device on the vehicledata bus 31, unless the network element or device on the implement databus 58 follows a certain security protocol, handshake, password and key,or another security measure. Further, in one embodiment, the gateway 29may encrypt communications to the vehicle data bus 60 and decryptcommunications from the vehicle data bus 60 if a proper encryption keyis entered, or if other security measures are satisfied. The gateway 29may allow network devices on the implement data bus 58 that communicatevia an open standard or third party hardware and software suppliers,whereas the network devices on the vehicle data bus 60 are solelyprovided by the manufacturer of the transferring vehicle (e.g.,self-propelled forage harvester) or those authorized by themanufacturer.

In FIG. 2, a first location determining receiver 42, a user interface44, a user interface processing module 26, and the gateway 29 arecoupled to the implement data bus 58, although in other embodiments suchelements or network devices may be connected to the vehicle data bus 60.Light controller 50 and spout controller 54 are coupled to the vehicledata bus 60. In turn, the light controller 50 and spout controller 54are coupled, directly or indirectly, to lights 14 on the transferringvehicle 91 and the spout 89 of the transferring vehicle 91 (e.g.,self-propelled forage harvester), respectively. Although the system ofFIG. 2 is well suited for use or installation on a self-propelled forageharvester (SPFH), the system of FIG. 2 may also be applied to harvestersor other heavy equipment.

FIG. 4A illustrates a top view of a transferring vehicle 91 and areceiving vehicle 79. Like reference numbers indicate like elements inFIG. 4A and FIG. 3. FIG. 4A shows a first imaging device 10 on the bodyof the transferring vehicle 91. The first imaging device 10 has a firstfield of view 477 indicated by the dashed lines. In FIG. 4A, the spout89 or spout end 87 is generally aligned over a central zone 83, centralregion or target area or grip pattern 82 of the storage unit 93 orcontainer 85 for unloading material from the transferring vehicle 91 tothe receiving vehicle 79. Similarly, the transferring vehicle 91 and thereceiving vehicle 79 are aligned in position as shown, and even as thevehicles 79, 91 move with coordinated headings or generally parallelheadings and with no or minimal relative velocity with respect to eachother.

In FIG. 5A, an optional second imaging device 12 may be mounted on thespout 87 of the transferring vehicle 91 with a second field of view 277,which may be slightly offset from, overlapped with, or aligned with thefirst field of view 477 to provide redundancy should the first imagingdevice 10 fail, malfunction, be unavailable, be unreliable, or providepoor quality image data. For example, the first imaging device 10 maynot operate reliably where it is obscured by dust, fog, salt, orair-born contaminants, or where it is exposed to inadequate ambientlight conditions or excessive glare from sunlight or reflected light. InFIG. 4A, the image processing module 18 can estimate the distance orrange from the first imaging device 10, the second imaging device 12, orboth to an object in the image, such as the spout 89, the spout end 87,the container perimeter 81, the level or profile of agriculturalmaterial in the container 85 (e.g., at various positions or coordinateswithin the container 85).

FIG. 4B illustrates a plan view of a transferring vehicle 91 and areceiving vehicle 79. Like reference numbers indicate like elements inFIG. 3, FIG. 4A and FIG. 4B. FIG. 4B shows a first imaging device 10only on the body of the transferring vehicle 91. The first imagingdevice 10 has a first field of view 477 indicated by the dashed lines.In FIG. 4B, the spout 89 or spout end 87 is generally aligned over acentral zone 83, central region or target area or grip pattern 82 of thestorage unit 93 or container 85 for unloading material from thetransferring vehicle 91 to the receiving vehicle 79. Similarly, thetransferring vehicle 91 and the receiving vehicle 79 are aligned inposition as shown, and even as the vehicles 79, 91 move with coordinatedheadings or uniform offset (e.g., Φ or φ; Δ).

FIG. 4C illustrates a view in a horizontal plane as viewed alongreference line 4C-4C in FIG. 4B. In one embodiment, the first imagingdevice 10 is mounted on the transferring vehicle 91 on a support 573(e.g., monopole with tilt or pan adjustment) to provide a downward fieldof view 677 or a down-tilted field of view.

In an alternate embodiment, the support 573 comprises an adjustable mastor telescopic mast that is controlled by a mast controller 674 toremotely adjust the height, tilt angle, down-tilt angle, rotation angle,or pan angle to provide reliable image data for processing by the imageprocessing module 18.

If the first imaging device 10 is elevated or mounted on thetransferring vehicle 91 sufficiently high with respect to the storageportion 93, the first imaging device 10 will have visibility or seconddownward field of view 677 into the storage portion 93 or container 85sufficient to observe and profile the surface (or height (z) versusrespective x, y coordinates in the container) of the agriculturalmaterial (e.g., grain) as the agricultural material fills the storageportion 85. The first imaging device 10 may be mounted on the roof ofthe transferring vehicle 91 facing or looking directly away from theside of the transferring vehicle 91 with the spout 89 for unloadingagricultural material.

In one illustrative configuration, consistent with the downward field ofview 677 the optical axis, perpendicular to respective lens, of thefirst imaging device 10 is tilted downward from generally horizontalplane at a down-tilted angle (ε) (e.g., approximately 10 to 25 degreesdownward). If a field of view or optical axis of the imaging device 10is tilted downward from a generally horizontal plane, there are severaladvantages. First, less of the sky is visible in the field of view ofthe imaging device 10 such the collected image data tends to have a moreuniform image intensity profile. The tilted configuration of the opticalaxis or axes (which is perpendicular to the lens of the imaging device10) is well suited for mitigating the potential dynamic range issuescaused by bright sunlight or intermediate cloud cover, for instance.Second, the bottom part of the storage portion 93 becomes more visiblein the image data to enable the recording of the image data related toone or more wheels of the storage portion 93. The wheel is a feature onthe storage portion 93 that can be robustly tracked by image processingtechniques. Third, tilting the stereo camera down may mitigate theaccumulation of dust and other debris on the lens or external window ofthe imaging device 10, 12.

FIG. 4D illustrates a two-dimensional representation of various possibleillustrative distributions of material in the container 85, consistentwith a view along reference line 4D in FIG. 4B. In one configuration,the y axis is coincident with the longitudinal axis or direction oftravel of the container, the z axis is coincident with the height ofmaterial in the container, and the x axis is perpendicular to thedirection of travel of the container, where the x, y and z axes aregenerally mutually orthogonal to each other.

In the chart of FIG. 5D, the vertical axis is the mean height (Z) 500 ofthe material in the container 85, the horizontal axis represents thelongitudinal axis (y) 502 of the container 85. The maximum capacity 504or container capacity is indicated by the dashed line on the verticalaxis. The front 512 of the container 85 is located at the origin,whereas the back 514 of the container 85 is located on the verticalaxis.

FIG. 4D shows three illustrative distributions of material within thecontainer 85. The first distribution is a bimodal profile 508 in whichthere are two main peaks in the distribution of material in thecontainer 85. The bimodal profile 508 is shown as a dotted line. Thebimodal profile 508 can occur where the spout angle adjustment isgoverned by an electro-hydraulic system with non-proportional valves.

The second distribution is the front-skewed modal profile 510 in whichthere is single peak of material toward the front of the container 85.The front-skewed modal profile 510 is shown as alternating long andshort dashes. The second distribution may occur where the volume orlength (y) of the container 85 is greater than a minimum threshold andwhere the relative alignment between the spout end 87 and the container85 is generally stationary during a substantial portion of unloading ofthe material.

The third distribution is the target profile 508 which may be achievedby following a suitable fill strategy as disclosed in this document. Forexample, during unloading, the spout angle may be adjusted to promoteuniform distribution of the agricultural material in the container 85.Further, the lateral offset (Δ) or fore/aft offset (Φ or φ) between thevehicles 79, 91 may be adjusted in accordance with a matrix (e.g., x, ycoordinate matrix of equidistant point locations of the transferringvehicle relative to a constantly spaced position point of the receivingvehicle) of relative unloading positions, particularly for longer orwider containers that cannot be uniformly filled from a single, relativeunloading point between the vehicles 79, 91.

FIG. 4E is a top view of a transferring vehicle 91 and a receivingvehicle 79, where the transferring vehicle 91 is aligned within a matrix500 of possible offset positions 502, 504 between the transferringvehicle 91 and receiving vehicle 79. Each offset position 502, 504 maybe defined in terms of a combination of a unique lateral offset (Δ) anda unique fore/aft offset (Φ or φ) between the vehicles 79, 91. As shown,the matrix 500 is a two-dimensional, 2×3 (2 columns by 3 rows) matrix ofpossible offset positions 502, 504. Although six possible matrixpositions 502, 504 are shown, in alternate embodiments the matrix 500may consistent of any number of possible offset positions greater thanor equal to two. Here, the transferring vehicle 91 occupies a currentoffset position 504 in the first column at the second row of the matrix500, whereas the other possible offset positions 502 are not occupied bythe transferring vehicle 91. As directed by any of the systems 11, 111,the imaging processing module 18, or the master controller 59 of thetransferring vehicle 91 can shift to any unoccupied or other possibleoffset positions 502 within the matrix 500 to promote or facilitate aneven distribution of agricultural material within the container 85 orstorage portion of the receiving vehicle 79. The spatial offset betweenthe transferring vehicle 91 and the receiving vehicle 79 may be adjustedin accordance with the matrix 500 or another matrix of preset positionsof spatial offset to promote even distribution of agricultural materialin the storage portion of the receiving vehicle 79, where any matrix isassociated with a unique, relative lateral offset (Δ) and a uniquefore/aft offset (Φ or φ) between the vehicles 79, 91.

In one embodiment of FIG. 4E, both the transferring vehicle 91 and thereceiving vehicle 79 may be moving forward at approximately the samevelocity and heading (e.g., within a tolerance or error of the controlsystems during harvesting), where the relative position of the receivingvehicle 79 is generally fixed or constant with respect to each position502, 504 in the matrix 500 that the transferring vehicle 91 can occupy.

In an alternate embodiment, the receiving vehicle 79 may be shown asoccupying a two dimensional matrix (e.g., 3×3 matrix, with three columnsand three rows) of possible offset positions, while the position of thetransferring vehicle 91 is generally fixed or constant with respect toeach position of matrix that the receiving vehicle 79 could occupy. Asdirected by any of the systems 11, 111 in the alternate embodiment, theimaging processing module 18 can shift to any unoccupied or otherpossible offset positions within the matrix to promote or facilitate aneven distribution of agricultural material within the container 85 orstorage portion 93 of the receiving vehicle 79.

In FIGS. 5A, 5B, 6A, and 6B, each of the blocks or modules may representsoftware modules, electronic modules, or both. Software modules maycontain software instructions, subroutines, object-oriented code, orother software content. The arrows that interconnect the blocks ormodules of FIG. 6 show the flow of data or information between theblocks. The arrows may represent physical communication paths or virtualcommunication paths, or both. Physical communication paths meantransmission lines or one or more data buses for transmitting, receivingor communicating data. Virtual communication paths mean communication ofdata, software or data messages between modules.

As illustrated in FIGS. 1 and 2, the first imaging device 10, the secondimaging device 12, or both, provide input of raw stereo camera images(or raw image data) to the image rectification module 101. FIG. 5A is ablock diagram that shows raw camera (monocular or stereo) processed byimage rectifier 101 to create rectified images for input into containeridentification module 20. Optional input into container identificationmodule 20 is spout localizer data 22, discussed in detail below. FIG. 5Bis a block diagram that shows raw camera images (monocular or stereo)processed by image rectifier 101 to create a rectified image. Therectified image will be processed by the disparity image generator 103to create ranges in the form of disparity data. Thereafter, rectifiedimages and disparity data are process by the spout localizer 22 withspout position data 1002. Output from spout localizer 22 is input intocontainer identification module 20. In an alternative embodiment, datafrom spout localizer 22 can be input into container identificationmodule 20 for a refinement in the material distribution in container orbin 85. Like reference numbers in FIGS. 1, 2, 5A, 6A, and 6B indicatelike elements.

FIG. 6A is a block diagram that shows raw camera images (monocular orstereo) processed by an image rectifier 101 to create rectified imagesfor input into spout localizer 22 for further processing with spoutposition data 1002 provided by vehicle model 1000. Output data fromspout localizer 22 can be input data for container identification module20.

FIG. 6B is a block diagram that shows raw camera images (monocular orstereo) processed by an image rectifier 101 to create rectified images.The rectified images will be processed by disparity generator 103 tocreate ranges in the form of disparity data. Thereafter, rectifiedimages and disparity data are processed by the spout localizer 22 alongwith spout position data 1002. The output data of spout localizer 22 canbe further processed by the container identification module 20. Theimage rectification module 101 provides image processing to thecollected image data or raw stereo images to reduce or remove radiallens distortion and image alignment required for stereo correspondence.The radial lens distortion is associated with the radial lenses of thefirst imaging device 10, the second imaging device 12, or both. Theinput of the image rectification module 101 is raw stereo image data,whereas the output of the image rectification module 101 is rectifiedstereo image data.

In one illustrative embodiment, the image rectifier 101 eliminates orreduces any vertical offset or differential between a pair of stereoimages of the same scene of the image data. Further, the imagerectification module can align the horizontal component (or horizontallines of pixels of the stereo images) to be parallel to the scan linesor common reference axis of each imaging device (e.g., left and rightimaging device) within the first and second imaging devices 10, 12. Forexample, the image rectifier 101 can remap pixels from initialcoordinates to revised coordinates for the right image, left image orboth to achieve registration of the images or rectified right and leftimages of the stereo image. The rectified image supports efficientprocessing and ready identification of corresponding pixels or objectswithin the image in the left image and right image of a common scene forsubsequent image processing.

In one configuration, the disparity image generator 103 applies a stereomatching algorithm or disparity calculator to collected stereo imagedata, such as the rectified stereo image data outputted by the imagerectifier 101. The stereo matching algorithm or disparity calculator maycomprise a sum of absolute differences algorithm, a sum of squareddifferences algorithm, a consensus algorithm, or another algorithm todetermine the difference or disparity for each set of correspondingpixels in the right and left image (e.g., along a horizontal axis of theimages or parallel thereto).

In an illustrative sum of the absolute differences procedure, the rightand left images (or blocks of image data or rows in image data) can beshifted to align corresponding pixels in the right and left image. Thestereo matching algorithm or disparity calculator determines a disparityvalue between corresponding pixels in the left and right images of theimage data. For instance, to estimate the disparity value, each firstpixel intensity value of a first subject pixel and a first sum of thefirst surrounding pixel intensity values (e.g., in a block or matrix ofpixels) around the first pixel is compared to each corresponding secondpixel intensity value of second subject pixel and a second sum of thesecond surrounding pixel intensity values (e.g., in a block or matrix ofpixels) around the second pixel. The disparity values can be used toform a disparity map or image for the corresponding right and left imagedata.

A container localizer estimates a distance or range from the firstimaging device 10, the second imaging device 12, or both to the pixelsor points lying on the container perimeter 81, on the container edge181, on the spout 89, on the spout end 87, or on any other linear edge,curve, ellipse, circle or object identified by the edge detector, thelinear Hough transformer, or both. For example, the image processingmodule 18 may use the disparity map or image to estimate a distance orrange from the first imaging device 10, the second imaging device 12, orboth to the pixels or points lying on the container perimeter 81, thecontainer edges 181, the container opening 83, in the vicinity of any ofthe foregoing items, or elsewhere.

In one embodiment, the container identification module 20 comprises: (1)an edge detector for measuring the strength or reliability of one ormore edges 181, or points on the container perimeter 81 in the imagedata; (2) a linear Hough transformer for identifying an angle and offsetof candidate linear segments in the image data with respect to areference point on an optical axis, reference axis of the one or moreimaging devices 10, 12; (3) a container localizer adapted to use spatialand angular constraints to eliminate candidate linear segments thatcannot logically or possibly form part of the identified linear segmentsof the container perimeter 81, or points on the container perimeter 81;and (4) the container localizer transforms the non-eliminated,identified linear segments, or identified points, into two or threedimensional coordinates relative to a reference point or reference frameof the receiving vehicle and harvesting vehicle.

The edge detector may apply an edge detection algorithm to rectifiedimage data from the image rectifier 101. Any number of suitable edgedetection algorithms can be used by the edge detector. Edge detectionrefers to the process of identifying and locating discontinuitiesbetween pixels in an image or collected image data. For example, thediscontinuities may represent material changes in pixel intensity orpixel color which defines boundaries of objects in an image. A gradienttechnique of edge detection may be implemented by filtering image datato return different pixel values in first regions of greaterdiscontinuities or gradients than in second regions with lesserdiscontinuities or gradients. For example, the gradient techniquedetects the edges of an object by estimating the maximum and minimum ofthe first derivative of the pixel intensity of the image data. TheLaplacian technique detects the edges of an object in an image bysearching for zero crossings in the second derivative of the pixelintensity image. Further examples of suitable edge detection algorithmsinclude, but are not limited to, Roberts, Sobel, and Canny, as are knownto those of ordinary skill in the art. The edge detector may provide anumerical output, signal output, or symbol, indicative of the strengthor reliability of the edges 181 in field. For example, the edge detectormay provide a numerical value or edge strength indicator within a rangeor scale or relative strength or reliability to the linear Houghtransformer.

The linear Hough transformer receives edge data (e.g., an edge strengthindicator) related to the receiving vehicle and identifies the estimatedangle and offset of the strong line segments, curved segments orgenerally linear edges (e.g., of the container 85, the spout 89, thespout end 87 and opening 83) in the image data. The estimated angle isassociated with the angle or compound angle (e.g., multidimensionalangle) from a linear axis that intercepts the lenses of the firstimaging device 10, the second image device 12, or both. The linear Houghtransformer comprises a feature extractor for identifying line segmentsof objects with certain shapes from the image data. For example, thelinear Hough transformer identifies line equation parameters or ellipseequation parameters of objects in the image data from the edge dataoutputted by the edge detector, or Hough transformer classifies the edgedata as a line segment, an ellipse, or a circle. Thus, it is possible todetect containers or spouts with generally linear, rectangular,elliptical or circular features.

In one embodiment, the data manager supports entry or selection ofcontainer reference data by the user interface 44. The data managersupports entry, retrieval, and storage of container reference data, suchas measurements of cart dimensions, by the image processing module 18 togive spatial constraints to the container localizer on the line segmentsor data points that are potential edges 181 of the cart opening 83.

In one embodiment, the angle estimator may comprise a Kalman filter oran extended Kalman filter. The angle estimator estimates the angle ofthe storage portion 93 (e.g., cart) of the receiving vehicle 79 to theaxis of the direction of travel of the propelled portion 75 (e.g.,tractor) of the receiving vehicle 79. The angle estimator (e.g., Kalmanfilter) provides angular constraints to the container localizer on thelines, or data points, that are potential edges 181 of the containeropening 83. In configuration, the angle estimator or Kalman filter iscoupled to the container localizer. The angle estimator filter outputs,or is capable of providing, the received estimated angle of the storageportion 93 relative to the axis of the direction of travel of thepropelling portion 75 of the vehicle.

The container localizer is adapted to receive measurements of dimensionsof the container perimeter 81 or the storage portion 93 of the vehicleto facilitate identification of candidate linear segments that qualifyas identified linear segments of the container perimeter 81. In oneembodiment, the container localizer is adapted to receive an estimatedangle of the storage portion 93 relative to the propelling portion 75 ofthe vehicle to facilitate identification of candidate linear segmentsthat qualify as identified linear segments of the container perimeter81. The container localizer uses spatial and angular constraints toeliminate candidate lines in the image data that cannot be possibly orlogically part of the container opening 83 or container edges 181, thenselects preferential lines (or data points on the container edge 81) asthe most likely candidates for valid container opening 83 (materialtherein) or container edges 181. The container localizer characterizesthe preferential lines as, or transformed them into, three dimensionalcoordinates relative to the vehicle or another frame of reference torepresent a container perimeter of the container 85.

In one embodiment, the spout localizer 22 comprises a spout classifierthat is configured to identify candidate pixels in the image data basedat least one of reflectivity, intensity, color or texture features ofthe image data (or pixels), of the rectified image data or raw imagedata, where the candidate pixels represent a portion of the spout 89 orspout end 87. The spout localizer 22 is adapted to estimate a relativeposition of the spout 89 to the imaging device based on the classified,identified candidate pixels of a portion of the spout 89. The spoutlocalizer 22 receives an estimated combine spout position or spout angle(α) relative to the mounting location of the imaging device, or opticalaxis, or reference axis of one or more imaging devices, based onprevious measurements to provide constraint data on where the spout 89can be located possibly.

The spout classifier applies or includes software instructions on analgorithm that identifies candidate pixels that are likely part of thespout 89 or spout end 87 based on expected color and texture featureswithin the processed or raw image data. For example, in oneconfiguration the spout end 87 may be painted, coated, labeled or markedwith a coating or pattern of greater optical or infra-red reflectivity,intensity, or luminance than a remaining portion of the spout 89 or thetransferring vehicle. The greater luminance, intensity or reflectivityof the spout end 87 (or associated spout pixels of the image data versusbackground pixels) may be attained by painting or coating the spout end87 with white, yellow, chrome or a lighter hue or shade with respect tothe remainder of the spout 89 or portions of the transferring vehiclewithin the field of view of the imaging devices 10, 12.

In one embodiment, the spout position estimator comprises a Kalmanfilter or an extended Kalman filter that receives input of previousmeasurements and container reference data and outputs an estimate of thespout position, spout angle, or its associated error. The spout positionestimator provides an estimate of the combine spout position, or spoutangle, or its error, relative to one or more of the following: (1) themounting location or pivot point of the spout on the transferringvehicle, or (2) the optical axis or other reference axis or point of thefirst imaging device 10, the second imaging device 12, or both, or (3)the axis associated with the forward direction of travel or the headingof the transferring vehicle. The Kalman filter outputs constraints onwhere the spout 89 or spout end 87 can be located, an estimated spoutposition, or a spout location zone or estimated spout position zone. Inone embodiment, the spout position estimator or Kalman filter is coupledto the spout localizer 22.

The spout localizer 22 takes pixels that are classified as belonging tothe combine auger spout 89 and uses a disparity image from disparityimage generator 103 to estimate the relative location of the spout tothe first imaging device 10, the second imaging device 12, or both, orreference axis or coordinate system associated with the vehicle.

FIG. 7 is a flow chart of a method for facilitating the unloading ofagricultural material from a vehicle or between a transferring vehicle91 and a receiving vehicle 79. The method of FIG. 7 may use one or moreof the following embodiments of the systems 11, 111 previously disclosedherein.

In step S902, the first imaging device 10 faces toward the storageportion of the receiving vehicle 79 (e.g., grain cart) and collectsfirst image data (e.g., first stereo image data, first monocular imagedata, or a right image of a stereo image). For example, the firstimaging device 10 may be mounted on the body of transferring vehicle 91facing the receiving vehicle 79 and facing the container 85. In oneembodiment, the first imaging device 10 has first field of view 77 (FIG.3) or view 477 (FIGS. 4A and 4B) of the storage portion of the receivingvehicle 79.

In an alternative embodiment, the first imaging device 10 comprises amonocular imaging device that provides a first image section (e.g., leftimage) of stereo image data of a scene or an object.

In step S904, where present, the optional second imaging device 12 facestoward the storage portion 93 of the receiving vehicle 79 (e.g., graincart) and collects second image data (e.g., second stereo image data,second monocular image data, or a left image of a stereo image). Forexample, the second imaging device 12 may be mounted on spout 89 of thetransferring vehicle 91 facing the receiving vehicle 79 (FIGS. 3 and4A). In one embodiment, the second imaging device 12 has a second fieldof view (177, 277) of the storage portion of the receiving vehicle,where the first field of view (77, 477) overlaps at least partially withthe second field of view (177, 277, respectively.

In an alternate embodiment, the second imaging device 12 comprises amonocular imaging device that provides a second image section (e.g.,right image) of stereo image data of a scene or an object, where theimage processing module 18 supports the creation of a stereo image froma combination of the first image section (of the first monocular imagingdevice) and the second image section with reference to the relativeposition and orientation of the first imaging device 10 and the secondimaging device 12.

In step S906, an image processing module 18 or a containeridentification module 20 identifies a container perimeter 81 of thestorage portion 93 in the collected image data (e.g., the first imagedata, the second image data or both), where the storage portion 93 hasan opening 83 inward from the container perimeter 81 for receipt of theagricultural material. Step S906 may be carried out in accordance withvarious techniques, which may be applied alternately or cumulatively.Under a first technique, the image processing module 18 or containeridentification module 20 may employ the following processes orsub-steps: (1) measuring a strength of one or more edges 181 in theimage data (raw and rectified image data); (2) identifying an angle andoffset of candidate linear segments in the image data with respect to anoptical axis, reference axis (e.g., direction of travel of thetransferring vehicle), or reference point indexed to one or more imagingdevices 10, 12; and (3) using spatial and angular constraints toeliminate identified candidate linear segments that cannot logically orpossibly form part of the identified linear segments of the containerperimeter, where the container identification module transforms theidentified linear segments into three dimensional coordinates relativeto a reference point or reference frame of the receiving vehicle and/orthe harvesting vehicle.

Under a second technique, the image processing module 18 or containeridentification module 20 may receive container reference data, ormeasurements of dimensions of the container perimeter 81 or the storageportion 93 of the vehicle, to facilitate identification of candidatelinear segments, or candidate data points, that qualify as identifiedlinear segments of the container perimeter 81.

Under the third technique, the image processing module 18 or containeridentification module 20 may receive an estimated angle of the storageportion 93 relative to the propelling portion 75 of the vehicle tofacilitate identification of candidate linear segments that qualify asidentified linear segments of the container perimeter 81.

Under a fourth technique, the image processing module 18 or containeridentification module 20 provides the received estimated angle of thestorage portion 93 relative to the propelling portion 75 of the vehicle.

In step S908, the image processing module 18 or a spout localizer 22identifies a spout 89 (or spout end 87) of the transferring vehicle 91in the collected image data. The image processing module 18 or the spoutlocalizer 22 may use various techniques, which may be appliedalternately or cumulatively. Under a first technique, the imageprocessing module 18 or the spout localizer 22 identifies candidatepixels in the image data (e.g., rectified or raw image data) based onexpected color and expected texture features of the image data, wherethe candidate pixels represent a portion of the spout 89 (e.g., combineauger spout) or spout end 87.

Under a second technique, the image processing module 18 or the spoutidentification module 22 estimates a relative position, or relativeangle, of the spout 89 or the spout end 87, to the imaging device basedon the classified, identified candidate pixels of a portion of the spout89.

Under a third technique, the image processing module 18 or the spoutidentification module 22 receives an estimated combine spout position,or spout angle, relative to the mounting location, optical axis,reference axis, or reference point of the imaging device 10, 12 based onprevious measurements to provide constraint data on where the spout 89can be located possibly.

Under a fourth technique, the image processing module 18 or spoutlocalizer 22 provides the estimated combine spout position, or estimatedspout angle, to the container identification module 20.

In step S910, the image data evaluator 25 or image processing module 18determines whether to use the first image data, the second image data orboth, based on an evaluation of the intensity of pixel data or ambientlight conditions. Step S910 may be carried out in accordance withvarious techniques, which may be applied alternately or cumulatively.

Under a first technique, where a first optical sensor 110 is associatedwith the respective first imaging device 10; the image data evaluator 25or image processing module 18 decides to use the first image data if thevariation in ambient light over a sampling time interval (e.g.,commensurate with a sampling rate of 1 to 120 samples per second) isless than or equal to a maximum ambient light variation, as measured bythe first optical sensor 110. Here, under the first technique the firstimage data is collected solely by the first imaging device 10. Abackground level, mean level, or mode level of variation in the ambientlight in the image data, a block of pixels in the first image data, oran object within the image data (e.g., spout, spout end, containerperimeter or container) may be gathered or tracked during operation ornormal operation of the systems 11, 111. In one embodiment, the maximumambient light level is set to be greater than the background level, meanlevel, or mode level. For example, the maximum ambient light level(e.g., within the visible light spectrum, near-infrared spectrum, orinfrared spectrum) is set to be greater by statistical measure (e.g.,approximately one to two standard deviations above the backgroundlevel), mean level or mode level, or a signal level difference betweenthe maximum ambient light level and the mean level of equal to orgreater than a threshold level (e.g., within a range of approximately 3decibels to 6 decibels).

Under a second technique, where a second optical sensor 112 isassociated with the second imaging device 12; the image data evaluator25 or image processing module 18 decides to use the second image data ifthe variation in ambient light over a sampling time interval (e.g.,commensurate with a sampling rate of 1 to 120 samples per second) isless than or equal to a maximum ambient light variation, as measured bythe second optical sensor 112. Here under the second technique, thesecond image data is collected solely by the second imaging device 12. Abackground level, mean level, or mode level of variation in the ambientlight in the second image data, a block of pixels in the image data, oran object within the image data (e.g., spout, spout end, containerperimeter or container) may be gathered or tracked during operation ornormal operation of the systems 11, 111. In one embodiment, the maximumambient light level is set to be greater than the background level, meanlevel, or mode level. For example, the maximum ambient light level(e.g., within the visible light spectrum, near-infrared spectrum, orinfrared spectrum) is set to be greater by statistical measure (e.g.,approximately one to two standard deviations above the backgroundlevel), mean level or mode level, or a signal level difference betweenthe maximum ambient light level and the mean level of equal to orgreater than a threshold level (e.g., within a range of approximately 3decibels to 6 decibels).

Under a third technique, the image processing module 18, or the imagedata evaluator 25 decides to use the first image data of the firstimaging device 10 if the variation in pixel intensity of a spout, aspout end, or a container in the first image over a sampling timeinterval is less than or equal to a maximum pixel intensity variation,as detected by the image processing module 18.

Under a fourth technique, the image processing module 18, or the imagedata evaluator 25 decides to use the second image data of the secondimaging device 12 if the variation in pixel intensity of a spout orspout end in the second image over a sampling time interval is less thanor equal to a maximum pixel intensity variation, as detected by theimage processing module 18.

Under a fifth technique, the image processing module 18, the image dataevaluator 25 is adapted for determining whether to use the first imagedata, the second image data or both, for the identification of thecontainer perimeter and the identifying of the spout (or spout end),based on pixel intensity in rejected image data being outside of adesired range or variation in pixel intensity during the sampling timeinterval, where the image processing module 18, the image data evaluator25 or image processing module 18 is configured to disable selectivelythe processing or use of the rejected image data that comprises aportion of the collected first image data or the second image data thatwould otherwise be corrupted by one or more of the following conditions:(1) excessive transient sunlight during sunrise, sunset, or excessivelight radiation from other sources (e.g., headlights from othervehicles), (2) transitory sunlight or cloud cover, (3) fog,precipitation or moisture, (4) shading (e.g., from vegetation, trees,buildings or plant canopies), (5) airborne dust or debris, (6)reflections of light (e.g., from polished, glossy or reflective surfacesof other machines or vehicles) or other lighting conditions that cantemporarily disrupt or interfere with proper operation of the imagingdevices 10, 12.

In step S912, the image processing module 18 or the alignment module 24determines the relative position of the spout 89, or the spout end 87,and the container perimeter 81 and for generating command data tomodulate the ground speed of the transferring vehicle 91 or repositionthe spout 89 or both in cooperative alignment such that the spout 89 (orspout end 87) is aligned with a central zone 83 of the containerperimeter 81. The image processing module 18 may use, retrieve or accesspreviously stored data, such as dimensional parameters related to thereceiving vehicle, the dimensional parameters comprising a distancebetween a trailer hitch and front wheel rotational axis of the storageportion 93. Such dimensional parameters may be entered via a userinterface 44 coupled to the vehicle data bus 60 or the image processingmodule 18, for example.

To execute step S912, the imaging processing module 18 may use firstlocation data of a first location determining receiver 42 on thetransferring vehicle 91 to determine relative position between the spoutand the container perimeter, and generate command data to modulate theground speed of the transferring vehicle 91 or reposition the spout 89or both in cooperative alignment such that the spout 89 is alignedwithin a central zone of the container perimeter 181 or a section ofgrid pattern 82.

In step S914, in a first configuration, the controller 59 or thepropulsion controller 40 modulates the ground speed of the transferringvehicle 91. In a second configuration, the vehicle controller 46 or thespout controller 54 repositions the spout 89. The rotation actuator 122(e.g., a servo-motor, electric motor, linear motor andlinear-to-rotational gear assembly, or electro-hydraulic device)controls the spout angle of the spout 89, or the spout end 87, withrespect to the direct of travel or another reference axis of thetransferring vehicle in response to alignment module 24 or the imageprocessing module 18 (e.g., smart unloading controller). In a thirdconfiguration, both the speed of the transferring vehicle and the spout89 is repositioned.

FIG. 8 illustrates the data flow and processing by the image processingmodule 18 from the raw images to the transferring vehicle commands. Thecomponents and modules have been discussed in detail above. The dashedlines represent optional steps and/or modules. Raw images are collectedby the imaging device 10, 12 (e.g. camera being either stereo ormonocular). Some embodiments of the present invention only require oneimaging device. Raw images are processed through the image rectifier 101to create rectified images. Rectified images are processed by the imagedata evaluator 25 to provide an image quality score for the rectifiedimage to determine if the image should be used in further processing bythe alignment module 24. Rectified images are also processed by thecontainer identification module 20 and material profile module 27.Rectified images can also be processed in conjunction with disparityimages by the spout localizer 22 when a disparity image generator 103 ispresent. Otherwise, spout localizer 22 will only use the data stored invehicle model 1000, which includes, but not limited to, data on thetransferring vehicle 91, dimensions of spout 89, and spout kinematicmodel. Spout localizer 22 also requires data about the vehicle stateinformation, which includes, but not limited to, transferring vehiclespeed, spout angle(s), auger drive on/off status, and relative GlobalPositioning Satellite position of receiving vehicle 79 if machinesynchronization is present. Spout localizer 22 output is input intocontainer identification module 20 and processed in conjunction withrectified images and disparity images (if provided) by containeridentification module 20 to determine container location and dimensions.Rectified images and disparity images (if provided) are processed inconjunction with container location and dimensions data from containeridentification module 20 by material profile module 27 to generate afill profile of the container 85. Alignment module 24 processes datagenerated by the container identification module 20, material profilemodule 27 in conjunction with the vehicle state information to generatevehicle commands such as transferring vehicle 91 speed/steering, spoutposition, auger drive on/off status, and speed/steering of the receivingvehicle 79 if machine synchronization is present to reposition the spoutend 87 over the appropriate open area of the container 85 for even,uniform distribution of the agricultural material in container 85.

While the disclosure has been described in detail and with reference tospecific embodiments thereof, it will be apparent to one skilled in theart that various changes and modifications can be made therein withoutdeparting from the spirit and scope of the embodiments. Thus, it isintended that the present disclosure cover the modifications andvariations of this disclosure provided they come within the scope of theappended claims and their equivalents.

The invention claimed is:
 1. A method for facilitating the transfer ofmaterial from a transferring vehicle having a material distribution endto a receiving vehicle having a bin to store the transferred material,the method comprising the steps of: a. obtaining image data from atleast one imaging device facing the bin, wherein the at least oneimaging device is mounted on the transferring vehicle; b. identifyingand locating the bin from the image data using an image processingmodule; c. identifying a spout from the image data using the imageprocessing module to determining a location and orientation of thespout; d. detecting a representation of the fill level or volumetricdistribution of the material in the bin; e. determining a position ofthe spout relative to the bin using an alignment module based on anidentified location of the bin and the identified location andorientation of the spout; f. aligning the spout over a current targetarea of the bin requiring the material by generating command data usedby at least one of a vehicle controller and a spout controller, whereinthe command data is based in part on the position of the spout relativeto the bin; g. determining subsequent target areas of the bin thatrequire material based on the representation of the fill level orvolumetric distribution of the material in the bin; h. transferring thematerial from the transferring vehicle to the current target area of thebin of the receiving vehicle; i. detecting when the current target areaof the bin is filled with the material; j. repeating steps f-i until thesubsequent target areas of the bin are filled; and k. terminating thetransfer of the material from the transferring vehicle to the receivingvehicle.
 2. The method according to claim 1, wherein the representationof the fill level or volumetric distribution of the material in the binis one-dimensional.
 3. The method according to claim 1, wherein therepresentation of the fill level or volumetric distribution of thematerial in the bin is two-dimensional.
 4. The method according to claim1, wherein the representation of the fill level or volumetricdistribution of the material in the bin is three-dimensional.
 5. Themethod according to claim 1, wherein the step of detecting therepresentation of the fill level or volumetric distribution of thematerial in the bin further comprises the steps of: receiving data froma distributed fill state sensor; and generating the representation ofthe fill level or volumetric distribution of the material in the binbased on the distributed fill state sensor.
 6. The method according toclaim 1, wherein the material is agricultural material.
 7. The methodaccording to claim 1, wherein the material is a mineral material.
 8. Themethod according to claim 1, wherein the step of detecting therepresentation of the fill level or volumetric distribution of thematerial in the bin further comprises the steps of: receiving rectifiedimage data from the at least one imaging device; generating disparityimage data based on the rectified image data; and generating therepresentation of the fill level or volumetric distribution of thematerial in the bin using range data based on the disparity image data.9. The method according to claim 8, wherein the step of receivingrectified image data from one or more imaging devices further comprisesreceiving a rectified image data from an imaging device on thetransferring vehicle, wherein there are no imaging devices on thereceiving vehicle.
 10. The method according to claim 8, wherein the stepof receiving rectified image data from the at least one imaging devicefurther comprises: receiving a first rectified image data from a firstimaging device on the transferring vehicle; and receiving a secondrectified image data from a second imaging device on the transferringvehicle, wherein there are no imaging devices on the receiving vehicle.11. The method according to claim 1, wherein the step of determiningsubsequent target areas of the bin that require material based on therepresentation of the fill level or volumetric distribution of thematerial in the bin comprises the steps of: developing a target zonematrix of the bin, wherein each target zone of the matrix is identifiedby a pre-established set of coordinates related to the bin; identifyingtarget zones of the matrix that are filled and not filled based on therepresentation of the fill level or volumetric distribution of thematerial in the bin; and determining the pre-established set ofcoordinates of a subsequent target zone to be filled and thepre-established set of coordinates of the current target zone over whichthe material distribution end is positioned.
 12. The method according toclaim 11, wherein the step of aligning the spout over a current targetarea of the bin requiring the material comprises the step of actuating acontrol mechanism to move the spout over the pre-established set ofcoordinates of a subsequent target zone to be filled.
 13. The methodaccording to claim 12, wherein the step of actuating a control mechanismto move the spout over the pre-established set of coordinates of asubsequent target zone to be filled comprises the steps of: establishinga current lateral spout angle of the spout; converting thepre-established set of coordinates between the current target zone andthe subsequent target zone to be filled into a change in the currentlateral spout angle to form a new lateral spout angle; transmitting thenew lateral spout angle to a spout controller of the transferringvehicle; and actuating the spout controller from the current lateralspout angle to the new lateral spout angle, whereby the spout is alignedover the subsequent target zone to be filled.
 14. The method accordingto claim 11, wherein the step of aligning the spout over a currenttarget area of the bin requiring the material comprises the step ofchanging a lateral offset between the receiving vehicle and thetransferring vehicle to move the spout from over the current target zoneto over the subsequent target zone to be filled.
 15. The methodaccording to claim 14, wherein the step of changing a lateral offsetbetween the receiving vehicle and the transferring vehicle comprises thestep of: establishing a current lateral offset between the receivingvehicle and the transferring vehicle; converting the pre-established setof coordinates between the current target zone and the subsequent targetzone to be filled into a change in the current lateral offset to form anew lateral offset; transmitting the new lateral offset to a steeringcontroller of the transferring vehicle; and steering the transferringvehicle to create the new lateral offset between the receiving vehicleand the transferring vehicle, whereby the spout is aligned the over thesubsequent target zone to be filled.
 16. The method according to claim11, wherein the step of aligning the spout over a current target area ofthe bin requiring the material comprises the step of changing a fore/aftoffset between the receiving vehicle and the transferring vehicle tomove the spout from over the current target zone to over the subsequenttarget zone to be filled.
 17. The method according to claim 16, whereinthe step of changing a fore/aft offset between the receiving vehicle andthe transferring vehicle comprises the step of: establishing an initialfore/aft offset between the receiving vehicle and the transferringvehicle; converting the pre-established set of coordinates between thecurrent target zone and the subsequent target zone to be filled into achange in the current fore/aft offset to form a new fore/aft offset;transmitting the new fore/aft offset to a propulsion controller of thetransferring vehicle, or a braking controller of the transferringvehicle; and accelerating or braking the transferring vehicle to createthe new fore/aft offset between the receiving vehicle and thetransferring vehicle, whereby the spout is aligned the over thesubsequent target zone to be filled.
 18. The method according to claim1, wherein the step of transferring the material from the transferringvehicle to the current target area of the bin of the receiving vehiclecomprises the step of actuating an auger.
 19. The method according toclaim 18, wherein the step of terminating the transfer of the materialfrom the transferring vehicle to the receiving vehicle comprises thestep of stopping the auger.
 20. The method according to claim 1, whereinthe step of transferring the material from the transferring vehicle tothe current target area of the bin of the receiving vehicle comprisesthe step of actuating the spout.
 21. The method according to claim 20,wherein the step of terminating the transfer of the material from thetransferring vehicle to the receiving vehicle comprises the step ofstopping the spout.
 22. The method according to claim 1, wherein thestep of detecting when the current target area of the bin is filled withthe material comprises the step of detecting a representation of thefill level or volumetric distribution of the material in the currenttarget zone of the bin.
 23. The method according to claim 22, whereinthe step of detecting a representation of the fill level or volumetricdistribution of the material in the current target zone of the binfurther comprises the steps of: receiving data from a distributed fillstate sensor; and generating the representation of the fill level orvolumetric distribution of the material in the current target zone ofthe bin based on the distributed fill state sensor data.
 24. The methodaccording to claim 22, wherein the step of detecting a representation ofthe fill level or volumetric distribution of the material in the currenttarget zone of the bin further comprises the steps of: receivingrectified image data from the at least one imaging device; generatingdisparity image data based on the rectified image data; and generatingthe representation of the fill level or volumetric distribution of thematerial in the bin using range data based on the disparity image data.25. The method according to claim 24, wherein the step of receivingrectified image data from the at least one imaging device furthercomprises receiving a rectified image data from an imaging device on thetransferring vehicle, wherein there are no imaging devices on thereceiving vehicle.
 26. The method according to claim 24, wherein thestep of receiving rectified image data from the at least one imagingdevice further comprises: receiving a first rectified image data from afirst imaging device on the transferring vehicle; and receiving a secondrectified image data from a second imaging device on the transferringvehicle, wherein there are no imaging devices on the receiving vehicle.